Global Data Analysis and the Fragmentation Problem in Decision Tree Induction

We investigate an inherent limitation of top-down decision tree induction in which the continuous partitioning of the instance space progressively lessens the statistical support of every partial (i.e. disjunc-tive) hypothesis, known as the fragmentation problem. We show, both theoretically and empirically, how the fragmentation problem adversely aaects predictive accuracy as variation r (a measure of concept dii-culty) increases. Applying feature-construction techniques at every tree node, which we implement on a decision tree inducer DALI, is proved to only partially solve the fragmentation problem. Our study illustrates how a more robust solution must also assess the value of each partial hypothesis by recurring to all available training data, an approach we name global data analysis, which decision tree induction alone is unable to accomplish. The value of global data analysis is evaluated by comparing modiied versions of C4.5rules with C4.5trees and DALI, on both artiicial and real-world domains. Empirical results suggest the importance of combining both feature construction and global data analysis to solve the fragmentation problem.